镜像站点 · 本页由第三方 GitHub 只读镜像提供,非 GitHub 官方站点,不接受任何登录或凭据输入。前往 github.com
Skip to content

fix(auto): do not divide by zero speech time in AutoModel.inference - #3896

Merged
LauraGPT merged 1 commit into
modelscope:mainfrom
sohailshk:fix/zero-rtf
Nov 14, 2026
Merged

LauraGPT merged 1 commit into
modelscope:mainfrom
sohailshk:fix/zero-rtf

Conversation

@sohailshk

Copy link
Copy Markdown
Contributor

Summary

AutoModel.inference computes time_escape / batch_data_time for its rtf statistic after the model has already returned its result. fsmn-vad reports batch_data_time == 0 when a call produces no feature frame, so the statistic raised ZeroDivisionError and the result was lost. This happens for one-shot inputs shorter than 400 samples (25 ms at 16 kHz) and for streaming VAD with chunk_size=10 or 20 ms, where the first chunk produces no frame. The progress-bar rtf_avg description has the same division. Both now report nan when no speech time was reported. Models that report -1 (no timing) keep the previous value (code reading).

Type of change

  • Bug fix

Validation

  • python -m compileall funasr examples tests
  • python -m pytest tests/test_auto_model_zero_speech_time.py -q: 2 failed on main (ZeroDivisionError at auto_model.py:986), 2 passed with the fix.
  • fsmn-vad (iic/speech_fsmn_vad_zh-cn-16k-common-pytorch), CPU:
    • generate(input=wav[:n]): main raises for n <= 399 and works from n = 400 (bisected); with the fix n = 100 and 399 return value: [].
    • Streaming loop (cache, is_final) over the first 20 s of example/vad_example.wav: on main chunk_size=10 and 20 raise at the first chunk, 30 ms and larger work. With the fix 10 and 20 ms give the same events as 200 ms ([[70, -1], [-1, 19980]]).
    • Batch generate(input=[wav_20s, wav[:100]]): main raises and returns nothing; with the fix the 20 s input gets [[70, 19980]].
    • AutoModel(model=..., vad_model=<fsmn-vad>) on 100 samples: main raises in the VAD step; with the fix it returns {'text': '', 'timestamp': []}. VAD finds no segment, so the second model is never called (fsmn-vad stood in for the ASR model).
  • Existing offline suites (test_load_audio_bytes, test_pcm_input_format, test_merge_vad_length, test_wav_frontend_no_lfr, test_fsmn_vad_*, test_dynamic_streaming_vad, test_punc_model_none, test_silero_vad_adapter, test_auto_model_ncpu): same results as on main.

User impact

Streaming VAD with 10 or 20 ms chunks (common audio-callback sizes), very short clips, and batches or VAD+ASR pipelines that contain such a clip no longer crash. A too-short input returns an empty result, as funasr_onnx does since #3822.

Notes for reviewers

Input shorter than one feature frame (about 25 ms) makes the model report
batch_data_time == 0, and the rtf statistic then raised ZeroDivisionError
after the model had already returned its (empty) result.
LauraGPT added a commit that referenced this pull request Nov 14, 2026
Preserve original #3896 ancestry. Scoped private suites and real native CUDA evidence; exact NumPy/KWS/MOSS CI accepted.

Signed-off-by: LauraGPT <zhifu.gzf@alibaba-inc.com>
@LauraGPT
LauraGPT merged commit cd76816 into modelscope:main Nov 14, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants